Indexing permissions as part of the Elastic Agent policy #169

aleksmaus · 2021-03-25T01:57:33Z

What does this PR do?

Implements the Indexing permissions as part of the Elastic Agent policy
#101

Adds a new field policy_output_permissions_hash into the Agent document that stores the hash of output_permissions from the policy.
Retrieves the policy from .fleet-policies upon the agent enrollment and uses the output_permissions default namespace in order to create the role descriptors for the output API key generation.
Checks the output_permissions upon every checking. If the hash doesn't match with the stored in the agent document the new key is generated.

This PR:

doesn't invalidate the old API key see the comment for the next iteration [Discuss] Indexing permissions as part of the Elastic Agent policy #101 (comment)
is not fully validated end-to-end with kibana for the output_permissions changes, pending on further kibana changes.

Should only be merged after the corresponding kibana changes:
elastic/kibana#94591

Why is it important?

Addresses: #101

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation

Related issues

Addresses [Discuss] Indexing permissions as part of the Elastic Agent policy #101

Screenshots

elasticmachine · 2021-03-25T02:02:53Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #169 updated
Start Time: 2021-03-29T13:25:33.463+0000
Duration: 4 min 11 sec
Commit: f625155

Test stats 🧪

Test	Results
Failed	0
Passed	66
Skipped	0
Total	66

Trends 🧪

ruflin · 2021-03-25T09:29:57Z

internal/pkg/policy/output_permissions.go

+		}
+
+		m := make(smap.Map)
+		m["index"] = idx


I hope we can get rid of this bit as it is planned that Kibana will add the index part to the output so we don't need to do the additional processing here.

Yeah, Alejandro reached out about the upcoming change. Might add more things here then.

blakerouse

Overall this change looks good. See my inline comment about fetching the policy from elasticsearch.

blakerouse · 2021-03-25T12:26:02Z

internal/pkg/dl/policies.go

@@ -79,3 +90,18 @@ func CreatePolicy(ctx context.Context, bulker bulk.Bulk, policy model.Policy, op
 	}
 	return bulker.Create(ctx, o.indexName, "", data)
 }
+
+// FindPolicyByID find policy by ID
+func FindPolicyByID(ctx context.Context, bulker bulk.Bulk, policyID string) (policy model.Policy, err error) {


The policy.Monitor keeps the latest revision of each policy in memory. It would be better to fetch the policy from there instead of going to elasticsearch, as fleet-server already has it cached in memory.

…or policy updates.

aleksmaus · 2021-03-25T13:34:53Z

Removed the output key generation upon enrollment. Now this is driven by the policy update pushes only, one code path.
Updated to the latest kibana changes, removed output permissions to the role descriptors transformation.

mergify · 2021-03-29T12:57:50Z

This pull request is now in conflicts. Could you fix it @aleksmaus? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b feature/indexing_permissions upstream/feature/indexing_permissions
git merge upstream/master
git push upstream feature/indexing_permissions

ruflin · 2021-03-29T18:32:28Z

@aleksmaus I see this got merged but elastic/kibana#94591 is still open. How is the current behaviour?

This reverts commit 81c6d16.

…" (#178) This reverts commit 81c6d16.

Indexing permissions as part of the Elastic Agent policy

2e87710

aleksmaus added the enhancement New feature or request label Mar 25, 2021

aleksmaus requested review from scunningham, blakerouse, nchaulet and ruflin March 25, 2021 01:57

ruflin reviewed Mar 25, 2021

View reviewed changes

blakerouse approved these changes Mar 25, 2021

View reviewed changes

Delay the output key generation. Now it is dirven by the policy monit…

d105fad

…or policy updates.

aleksmaus mentioned this pull request Mar 25, 2021

Add policy_output_permissions_hash field to .fleet-agents mapping elastic/kibana#95410

Merged

Merge branch 'master' into feature/indexing_permissions

f625155

aleksmaus merged commit 81c6d16 into elastic:master Mar 29, 2021

aleksmaus added a commit that referenced this pull request Mar 29, 2021

Revert "Indexing permissions as part of the Elastic Agent policy (#169)"

45410b4

This reverts commit 81c6d16.

aleksmaus mentioned this pull request Mar 29, 2021

Revert "Indexing permissions as part of the Elastic Agent policy" #178

Merged

aleksmaus added a commit that referenced this pull request Mar 29, 2021

Revert "Indexing permissions as part of the Elastic Agent policy (#169)…

4a15fa5

…" (#178) This reverts commit 81c6d16.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Indexing permissions as part of the Elastic Agent policy #169

Indexing permissions as part of the Elastic Agent policy #169

aleksmaus commented Mar 25, 2021

elasticmachine commented Mar 25, 2021 •

edited

Loading

Build stats

Test stats 🧪

Trends 🧪

ruflin Mar 25, 2021

aleksmaus Mar 25, 2021

blakerouse left a comment

blakerouse Mar 25, 2021

aleksmaus commented Mar 25, 2021 •

edited

Loading

mergify bot commented Mar 29, 2021

ruflin commented Mar 29, 2021

Indexing permissions as part of the Elastic Agent policy #169

Indexing permissions as part of the Elastic Agent policy #169

Conversation

aleksmaus commented Mar 25, 2021

What does this PR do?

Why is it important?

Checklist

Related issues

Screenshots

elasticmachine commented Mar 25, 2021 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

ruflin Mar 25, 2021

Choose a reason for hiding this comment

aleksmaus Mar 25, 2021

Choose a reason for hiding this comment

blakerouse left a comment

Choose a reason for hiding this comment

blakerouse Mar 25, 2021

Choose a reason for hiding this comment

aleksmaus commented Mar 25, 2021 • edited Loading

mergify bot commented Mar 29, 2021

ruflin commented Mar 29, 2021

elasticmachine commented Mar 25, 2021 •

edited

Loading

aleksmaus commented Mar 25, 2021 •

edited

Loading